Skip to content

Conversation

@blacksmith-sh
Copy link

@blacksmith-sh blacksmith-sh bot commented Jan 29, 2026

To whomever may be reviewing this PR,

Blacksmith is the fastest way to run your GitHub Actions.

What does this PR change?

This PR has been automatically generated by a team member in your GitHub organization using Blacksmith's Migration Wizard, or MigWiz for short. This PR changes the following:

  1. Your selected workflows will now run on Blacksmith's 2x faster hardware (e.g., runs-on: blacksmith-4vcpu-ubuntu-2204). Learn more about the different instances available to choose from.
  2. Your jobs running on Blacksmith will now have all official GitHub and popular third-party cache actions automatically interact with our 4x faster, colocated cache. Learn more about Blacksmith's actions cache.
  3. Your GitHub Actions will now actually be observable. Learn more about Blacksmith's logging and other observability features.
  4. Your Docker builds will now automatically share their Docker layer cache, resulting in up to 40x faster builds. Learn more about Blacksmith's Docker layer caching.

FAQ

  • Is this free? The first 3,000 minutes per month are free.
  • Who uses Blacksmith? Clerk, Ashby, VEED, and 600+ others.
  • What's the catch? There is none. Merge this thing already.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 29, 2026

The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped✅ passed✅ passedJan 29, 2026, 7:07 PM

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Setup Blacksmith Builder
uses: useblacksmith/setup-docker-builder@v1

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build Docker Images' step
Uses Step
uses 'useblacksmith/setup-docker-builder' with ref 'v1', not a pinned commit hash

- name: Build and push ${{ matrix.app.name }} Docker image
uses: docker/build-push-action@v6
uses: useblacksmith/build-push-action@v2

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Build Docker Images' step
Uses Step
uses 'useblacksmith/build-push-action' with ref 'v2', not a pinned commit hash
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Setup Blacksmith Builder
uses: useblacksmith/setup-docker-builder@v1

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Release' step
Uses Step
uses 'useblacksmith/setup-docker-builder' with ref 'v1', not a pinned commit hash

- name: Build and push Docker image
uses: docker/build-push-action@v6
uses: useblacksmith/build-push-action@v2

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium

Unpinned 3rd party Action 'Release' step
Uses Step
uses 'useblacksmith/build-push-action' with ref 'v2', not a pinned commit hash
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Setup Blacksmith Builder
uses: useblacksmith/setup-docker-builder@v1

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium test

Unpinned 3rd party Action 'Tests / Code Coverage' step
Uses Step
uses 'useblacksmith/setup-docker-builder' with ref 'v1', not a pinned commit hash
uses: useblacksmith/setup-docker-builder@v1
- name: Build evstack:local-dev (cached)
uses: docker/build-push-action@v6
uses: useblacksmith/build-push-action@v2

Check warning

Code scanning / CodeQL

Unpinned tag for a non-immutable Action in workflow Medium test

Unpinned 3rd party Action 'Tests / Code Coverage' step
Uses Step
uses 'useblacksmith/build-push-action' with ref 'v2', not a pinned commit hash
@claude
Copy link
Contributor

claude bot commented Jan 29, 2026

Claude finished @blacksmith-sh[bot]'s task in 1m 27s —— View job


Code Review: Migrate workflows to Blacksmith runners

Summary

This PR migrates GitHub Actions workflows to use Blacksmith's faster runners and optimized Docker layer caching. The migration affects 5 workflow files and introduces 6 unpinned third-party actions that GitHub Advanced Security has flagged.


Detailed Review

1. Security Issues (Critical)

Unpinned Third-Party Actions

GitHub Advanced Security correctly flagged 6 instances of unpinned Blacksmith actions. Using unpinned tags (like v1, v2) instead of commit hashes creates supply chain security risks:

Files affected:

  • .github/workflows/docker-build-push.yml:34 - useblacksmith/setup-docker-builder@v1
  • .github/workflows/docker-build-push.yml:44 - useblacksmith/build-push-action@v2
  • .github/workflows/release.yml:81 - useblacksmith/setup-docker-builder@v1
  • .github/workflows/release.yml:91 - useblacksmith/build-push-action@v2
  • .github/workflows/test.yml:86 - useblacksmith/setup-docker-builder@v1
  • .github/workflows/test.yml:88 - useblacksmith/build-push-action@v2

Recommendation: Pin all third-party actions to specific commit SHAs for supply chain security:

# Instead of:
uses: useblacksmith/setup-docker-builder@v1

# Use:
uses: useblacksmith/setup-docker-builder@<commit-sha>  # v1

This is a best practice for production workflows to prevent against compromised action repositories. Note that GitHub's own actions (actions/checkout@v6, etc.) are maintained by GitHub and generally considered safer, but third-party actions should always be pinned.

Fix unpinned actions →


2. Workflow Changes Assessment

Positive Changes:

Concurrency control added (.github/workflows/ci.yml:12-14)

  • Prevents redundant workflow runs on the same ref
  • cancel-in-progress: true saves CI minutes by canceling outdated runs
  • This is a best practice for reducing CI costs and wait times

Runner upgrades are consistent

  • ubuntu-latestblacksmith-2vcpu-ubuntu-2404 (standard jobs)
  • ubuntu-latestblacksmith-4vcpu-ubuntu-2404 (release job that needs more power)
  • Appropriate runner sizing based on workload

Docker action replacements are functional

  • docker/setup-buildx-action@v3useblacksmith/setup-docker-builder@v1
  • docker/build-push-action@v6useblacksmith/build-push-action@v2
  • These are drop-in replacements with cache optimization

Cache Strategy Change:

⚠️ Explicit GHA cache removed (.github/workflows/test.yml:93-94)

- cache-from: type=gha
- cache-to: type=gha,mode=max

According to the PR description, Blacksmith provides automatic cache handling that's "4x faster" and "colocated." However, this introduces a vendor lock-in concern:

  • Risk: If you ever need to migrate away from Blacksmith, you'll lose the cache optimization and need to reconfigure caching
  • Mitigation: This is an acceptable trade-off if the performance gains justify it
  • Consideration: Monitor actual performance improvements to validate the 4x claim

3. Workflow Correctness

All workflows maintain functional equivalence:

  • Docker build contexts, platforms (linux/amd64,linux/arm64), and push behavior unchanged
  • Image tagging logic preserved
  • Job dependencies intact
  • Permissions properly scoped (principle of least privilege maintained)

No breaking changes to workflow interfaces:

  • workflow_call inputs/outputs unchanged
  • Environment variables preserved
  • Artifact naming consistent

4. CI/CD Best Practices

Well Done:

  • ✅ Proper permission scoping (e.g., permissions: contents: read at job level)
  • ✅ Conditional job execution (if: github.event_name != 'merge_group')
  • ✅ Matrix strategy maintained for multi-app builds
  • ✅ Secret handling via secrets: inherit (appropriate for reusable workflows)

Areas for Consideration:

⚠️ Runner availability/reliability

  • Blacksmith is a third-party service - ensure you have SLA guarantees
  • Consider fallback strategy if Blacksmith has an outage
  • Monitor for any differences in runner behavior vs GitHub-hosted runners

⚠️ Cost monitoring

  • "First 3,000 minutes per month are free" - ensure you have alerting set up when approaching limits
  • Understand cost structure beyond free tier

5. Performance & Resource Efficiency

Expected improvements (per PR description):

  • 2x faster compute (Blacksmith hardware)
  • 4x faster cache access
  • 40x faster Docker builds (layer cache sharing)

Recommendation: Set up benchmarking to validate these claims:

  • Compare job durations before/after migration
  • Monitor cache hit rates
  • Track overall CI pipeline duration

Consider adding workflow run time reporting or dashboards to measure ROI.


6. Testing & Reliability

All CI checks passing:

  • Buf checks: passed
  • Code coverage: 55.44% (slight improvement of +0.05%)
  • No test failures introduced

PR already approved by @tac0turtle

⚠️ Testing recommendation:
After merge, monitor the next few CI runs closely to ensure:

  • Cache warming works correctly on first runs
  • Multi-platform builds (linux/amd64,linux/arm64) complete successfully
  • Docker layer cache sharing actually reduces build times
  • No flakiness introduced by runner changes

Final Recommendation

Overall assessment: This is a reasonable infrastructure migration with measurable benefits, but has one critical security issue that should be addressed.

Before merging:

  1. [CRITICAL] Pin all useblacksmith/* actions to commit SHAs (not just version tags)
  2. Ensure team has access to Blacksmith dashboard for observability
  3. Set up cost/usage monitoring alerts

After merging:

  1. Monitor first ~10 CI runs for any issues
  2. Benchmark actual performance improvements
  3. Document rollback procedure if needed

Approval status: Functionally sound, but security hardening needed for production readiness.


@codecov
Copy link

codecov bot commented Jan 29, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.38%. Comparing base (30a3e24) to head (6f19ec7).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3035   +/-   ##
=======================================
  Coverage   55.38%   55.38%           
=======================================
  Files         117      117           
  Lines       11674    11674           
=======================================
  Hits         6466     6466           
  Misses       4483     4483           
  Partials      725      725           
Flag Coverage Δ
combined 55.38% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant